Automated Trait Scores for "GRE"® Writing Tasks. Research Report. ETS RR-15-15

Autor/inn/en	Attali, Yigal; Sinharay, Sandip
Titel	Automated Trait Scores for "GRE"® Writing Tasks. Research Report. ETS RR-15-15
Quelle	In: ETS Research Report Series, (2015), (16 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2330-8516
Schlagwörter	Scores; Computer Assisted Testing; Computer Software; Grammar; Factor Analysis; College Entrance Examinations; Graduate Study; Writing Tests; Scoring; Reliability; Comparative Analysis; Essays; Construct Validity; Content Analysis; Language Usage; Weighted Scores; Vocabulary; Language Styles; Correlation; Goodness of Fit; Regression (Statistics); Task Analysis; Prediction; Questionnaires; Graduate Record Examinations + Suchen Sie Ihr Suchwort? Grammatik; Faktorenanalyse; Aufnahmeprüfung; Aufbaustudium; Graduiertenstudium; Hauptstudium; Writing test; Schreibtest; Bewertung; Reliabilität; Essay; Aufsatzunterricht; Inhaltsanalyse; Sprachgebrauch; Wortschatz; Sprachstil; Korrelation; Regression; Regressionsanalyse; Aufgabenanalyse; Vorhersage; Fragebogen
Abstract	The "e-rater"® automated essay scoring system is used operationally in the scoring of the argument and issue tasks that form the Analytical Writing measure of the "GRE"® General Test. For each of these tasks, this study explored the value added of reporting 4 trait scores for each of these 2 tasks over the total e-rater score. The 4 trait scores are word choice, grammatical conventions, fluency and organization, and content. First, confirmatory factor analysis supported this underlying structure. Next, several alternative ways of determining feature weights for trait scores were compared: weights based on regression parameters of the trait features on human scores, reliability of trait features, and loadings of features from factor analytic results. In addition, augmented trait scores, based on information from other trait scores, were also analyzed. The added value of all trait score variants was evaluated by comparing the ability to predict a particular trait score on one task from either the same trait score on the other task or the e-rater score on the other task. Results supported the use of trait scores and are discussed in terms of their contribution to the construct validity of e-rater as an alternative essay scoring method. (As Provided).
Anmerkungen	Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01